Tactical Decision Making for Lane Changing with Deep Reinforcement Learning

In this paper we consider the problem of autonomous lane changing for self driving cars in a multi-lane, multi-agent setting. We present a framework that demonstrates a more structured and data efficient alternative to end-to-end complete policy learning on problems where the high-level policy is hard to formulate using traditional optimization or rule based methods but well designed low-level controllers are available. The framework uses deep reinforcement learning solely to obtain a high-level policy for tactical decision making, while still maintaining a tight integration with the low-level controller, thus getting the best of both worlds. This is possible with Q-masking, a technique with which we are able to incorporate prior knowledge, constraints and information from a low-level controller, directly in to the learning process thereby simplifying the reward function and making learning faster and efficient. We provide preliminary results in a simulator and show our approach to be more efficient than a greedy baseline, and more successful and safer than human driving.